Surgical site displacement tracking

ABSTRACT

A method comprises providing a current state image and at least one reference image, taken from a similar angle range. The image and the at least one reference image are superimposed and a visual representation visualizing the relation between the image and the reference image is provided in order to track displacements of the bone during subsequent operation steps. A system is provided which can use the image data to track displacements and determine deviation from a current state of the elements in question to a target state.

BACKGROUND OF THE INVENTION

The invention relates to the field of computer based assistance ofsurgery. In particular, the invention relates to a method and acorresponding system of automatic image processing including aninterpretation of the content of for example two subsequently takenx-ray images. The method may be implemented as a computer programexecutable on a processing unit of a suitable device.

In every surgery where intraoperative imaging is used it is achallenging task to accurately perform the necessary steps of aprocedure of treating a bone fracture. In particular displacement of abone, a bone part or a bone fragment can easily occur in surgery stepsrequiring exertion of pressure or force. Such surgery step may, forinstance, be a final lag screw placement, requiring exertion of greaterforce by a surgeon as the friction between screw and bone gets larger.

There is the risk that a displacement, such as a translation or arotation of a bone or a bone fragment, is not detected by a surgeon.

BRIEF SUMMARY OF THE INVENTION

It may therefore be seen as an object of the invention to provide amethod and device for more efficiently assisting in performing asurgical procedure.

The mentioned objects are solved by the subject-matter of each of theindependent claims. Further embodiments are described in the respectivedependent claims and with reference to the drawings.

According to a first aspect of the invention a method for displacementtracking during the positioning and fixing of an implant in a bone, forexample a hip bone, is provided. The method comprises the step ofproviding an image of the bone and providing at least one referenceimage of the bone. The image may be the most current or latest x-rayimage taken during the surgical procedure. In a further method-step theimage and the at least one reference image are superimposed, whereinsuch superimposing requires that the image and the reference image aretaken from a similar angle range. Subsequently, a relationship betweenthe image and the at least one reference image is visualized in order totrack a potential displacement of the bone or a bone fragment relativeto an implant.

It would be highly beneficial to better control and point out undesireddisplacements of a bone or a bone fragment relative to an implant infinal placement and fixing of the implant. During positioning and fixingof an implant, forces have to be exerted to achieve a good fixation ofthe implant in, e.g., a bone or bone portion. Force exertion may,however, lead to unwanted and undesired displacements of the bone or theimplant. Such displacement may be overlooked or may not be identified bya surgeon, when the latter only considers one image of the surgicalarea, e.g. after final lag screw placement. The present inventionprovides a better control of undesired displacements by providing avisual representation of a relationship between an image and a referenceimage. The visualization of the relationship can help a surgeon or userto detect or better identify undesired displacements, which were causedin a surgery step after taking the reference image and before taking thecurrent intraoperative image.

The image and/or the reference image may be an x-ray image. Further, thereference image can be provided by a database and/or can be an earlieracquired image of the bone taken from a similar angle as the image. Thestep of superimposing can comprise a computer internal superpositionand/or a visual superposition of the image and the at least onereference image. Superimposing of the image and the at least onereference image can also comprise a matching of the scale and/or portionof the image and the at least one reference image.

The notion “similar angle range” denotes that the deviations between theangles defining the perspective from which the image is taken and theperspective from which the at least one reference image is taken aresmall, for instance between 0° and 5°. For such small deviations, theperspectives or viewpoints of the image and the at least one referenceimage are sufficiently close to each other to allow for a superimposingof the image and the at least one reference image. In this case,displacements of a bone relative to the position of an implant orrelative to a background structure can be detected by comparison of theimage and the at least one reference image. In particular, there is noadditional need to process or adjust the image or the reference imagebeforehand by a rotation. A background structure in the abovecontext—more generally in context of the invention—can be an anatomicregion or part of an anatomic region depicted/visible in the imageand/or the reference image, which is not displaced e.g. by a rotation ora lateral or transversal movement during positioning and fixing of animplant in a bone.

Detection of the angles from which the image and the reference image aretaken, respectively, can be performed automatically—e.g. by a suitablesoftware—by, for instance, relying on detection and identification ofthe position and orientation of a reference object in the image and thereference image. By determination of these angles it can be ensured thatthe image and the at least one reference image are indeed taken from asimilar angle range, with deviations between corresponding angles in theimage and the at least one reference image between, for instance, 0° andmaximally 5°. Thereby, a reference object can be an object directlyattached to an anatomical structure, e.g. being in contact with an outersurface of a body part of interest, or at least part of an implant, e.g.given by radiopaque spheres attached to an implant or attached to adrilling tool for fixing a bone screw. Instead or in addition to thedetection of the view-point angles of the images based on a referenceobject visible in the respective image, also a non-rotating part of theanatomy visible in the images can be used in order to automatically—byimage processing algorithms—determine the angle(s) under which theimages are taken. By an automatic determination of the angles, fromwhich the images are taken, and, consequently, an automaticdetermination of the deviation between corresponding angles related tothe image and the at least one reference image it can be ensuredautomatically, that the angle range of the at least two images issimilar. As a consequence, if the deviation between corresponding anglesin the image and the at least one reference image would exceed e.g. adeviation of 5° a warning could be generated and further method stepswould not be performed unless a new/another image with a similar anglerange as the reference image would be available. In this way it isautomatically ensured that the deviation between the angles under whichthe image and the at least one reference image are taken is smallenough.

As an example, displacement of the bone or a bone fragment can occur infinal lag screw placement—that is during the last few turns of a lagscrew before the screw is brought in its final position. In this case,the friction between the screw and the bone can get so high that, forinstance, proximal fragments could rotate and thereby be displaced fromtheir desired position. The risk for such displacement is higher themore unstable a fracture.

According to an embodiment of the invention, the visualization of therelationship between the image and the at least one reference imagecomprises visually representing the image and the least one referenceimage in juxtaposition.

Visualization can be performed by an imaging unit, the imaging unitcomprising a display for visual representation. The imaging unit can bepart of an imaging system comprised in a medical device. The imagingunit can, however, also be a separate imaging unit to which the imageand the at least one reference image are provided.

According to an embodiment of the invention, the most current image andthe at least one reference image may be visually represented on adisplay in juxtaposition, for instance side by side or one above theother.

According to an embodiment of the invention, the visualization of therelationship between the image and the at least one reference imagecomprises a switching between a visual representation of the image and avisual representation of the at least one reference image.

This implies that the image and the at least one reference image arevisualized, e.g. displayed on a computer display, one at a time and thata surgeon or user can switch or toggle between the two visualizations.Such switching or toggling can also be done automatically, e.g. bysubsequently showing the image and the at least one reference image fora predetermined time period, respectively.

According to an embodiment of the invention, the step of visualizingcomprises a combined visual representation of the current image and theat least reference image. In other words, the image and the at least onereference image are simultaneously visualized in superposition.

In this case it is preferred that the image and the at least onereference image have a color marking of objects or devices in therespective image. In addition to or instead of a color marking, theimage and the at least one reference image can have a marking byhatching or shading. The color marking and/or hatching and/or shadingcan be the same or can be different in the image and the at least onereference image. As an example, the lag screw in the image may becolored differently from the lag screw in the at least one referenceimage. In this case, the different colors can help to indicate to asurgeon a possible displacement of lag screw in the image with respectto the at least one reference image.

According to an embodiment of the invention, the step of visualizing isdone for a series of images of the bone, the series of images comprisingthe image and a set of reference images.

Depending on the workflow, more than one reference image may beavailable in a database or might have been taken in a previous surgerystep. In this case the before and hereinafter described visualizationmay comprise a visualization of the current image and all availablereference images, or a selectable or predetermined subset of thereference images. Selection of the reference images to be visualized canbe done by hand or according to a timeline. For instance, only a set ofthe latest taken reference images or reference images taken in apredetermined time period can be selected automatically.

According to an embodiment of the invention, the most current image andthe at least one reference image comprise at least one reference body.The superimposing of the image and the at least one reference image canthen be based on the reference body in the image and the at least onereference image.

A reference body can be one of a bone, a bone portion, a bone shape, animplant or an additional reference object.

An additional reference object may be directly attached to an anatomicalstructure, e.g. may be in contact with an outer surface of a body partof interest. The reference object may also be indirectly coupled to ananatomical structure, for example via a handling tool for inserting animplant. On the other hand, the reference object may be at least a partof an implant. In other words, an implant which is adapted to be fixedat or in a bone may comprise elements which can be identified in animage of the bone or at least a section of the bone so that geometricalaspects may be determined based on the identified elements. For example,the elements may define points so that two elements may define a line oran axis, or the elements may define a contour so that a center axis maybe determined.

According to an embodiment of the invention, the method furthercomprises the step of detecting an outline of the bone in the image andin the at least one reference image. The method-step of superimposingcan then be based or rely on the detected outline in the images. Inother words, the outline can be used in the process of superimposing andmatching of the most current image and the at least one reference image.

For outline detection, grayscale values of pixels may, for instance, beidentified and compared with adjacent pixel grayscale values so as todetect contour lines or points in the image and the at least onereference image, respectively. The contour lines can than used todetermine or define an outline of an object, e.g. a bone or an implant,in the image and the at least one reference image.

According to an embodiment of the invention, the method-step ofsuperimposing the image and the at least one reference image is based ona pixel wise comparison of the pixels in the image and the referenceimage. Alternatively, an area wise comparison or a comparison ofclusters of pixels in the image and the reference image can beperformed.

Hence, the step of superimposing can generally rely on a differenceanalysis of the images.

According to an embodiment of the invention, the step of superimposingthe current image and the at least one reference image comprises only anadjusting of a section and/or a scale of the image and the at least onereference image. In addition, or alternatively, adjusting of a portionof the image and/or the at least one reference image may be comprised inthe step of superimposing the image and the at least one referenceimage.

The image and the reference image can be taken from the same angle, butfrom differing distances. In the latter case, the scale of the image andthe reference image has to be adjusted before a visual representation ofboth images for further visual comparison by a user is provided. Thismight comprise choosing of a portion of the image or a portion of the atleast one reference image to be superimposed, as either the image or theat least one reference image might show a larger region. In another casethe section or detail of the image and the reference image may differ,while they are taken from the same angle and from the same distance.Then the common overlap or intersection of both images should bedetermined in the step of superimposing and the further imageprocessing—comprising the visualization—refers to the respective detailof the image and the reference image which is comprised in both theimage and the reference image. Determination of the common overlap orintersection of both images may in particular be based on determinationof the position and orientation of a reference body or parts of areference body visible in both images.

According to an embodiment of the invention, the angle range between theimage and the at least one reference image is between 0° and 5°,preferably 0° and 4°, more preferably 0° and 3°. For this angle range,the perspectives or viewpoints of the image and the at least onereference image are sufficiently close to allow for a superimposing. Inother words, for an angle range between 0° and 5°, the current image andthe at least one reference image have a common viewing angle of asurgery area or region of interest. Given the common viewing angle,displacements of a bone relative to the position of an implant orrelative to a background structure, such as surrounding tissue, can bedetected by comparison of the image and the at least one reference imagewithout the beforehand need to process or adjust the image or thereference image by a rotation. A comparison of the image and the atleast one reference image relies on superimposing the image and the atleast one reference image.

A surgery area or region of interest in this context may be a bone andthe bone-surrounding area of a patient.

According to an embodiment of the invention, in a further method-step adistinction between primary structures and non-primary structures in theimage and the reference image is performed. Non-primary structures aresubsequently removed and only the primary structures are depicted orshown in the visual relationship of the image and the reference image.Thereby, a primary structure can be at least one of a bone, a boneshape, a bone fragment, an implant, a reference body and/or anadditional reference object. An example for a non-primary structuremight be surrounding tissue structure which might not be of particularrelevance in lag screw placement.

In addition to or instead of any of the aforementioned or belowdescribed method steps it can be advantageous to detect or determine ina method step those objects in the image and the at least one referenceimage, which could potentially be displaced during positioning andfixing of an implant in a bone, and whose displacement is undesired. Forinstance, displacement of a bone or part of a bone during final lagscrew placement is undesired and a corresponding displacement should bevisualized in order to provide an indication and/or correspondingwarning to a surgeon. However, also e.g. the tip of a lag screw will bedisplaced during final lag screw placement. The correspondingdisplacement of the lag screw can be desired and displacement of onlythe lag screw may not be particularly marked and/or colored in avisualization in the image and the at least one reference image.Generally, displacements of anatomical parts in the images may bevisualized, e.g. by different color marking or in any other of the aboveand below described ways, but a desired displacement of the lag screwwill not be particularly marked. A desired displacement of a lag screwmay be a displacement where the tip of the lag screw is moved forward indirection of the longitudinal axis of the screw. An undesireddisplacement of a lag screw, however, might be a displacement where thetip of the screw is bent away from the direction given by thelongitudinal axis of the screw, and such lag screw displacement might bevisualized. In order to automatically detect those object in the imageand the at least one reference image whose displacement shall beindicated to a surgeon by means of the method described herein, adatabase with such objects—comprising the form and shape of theseobjects—can be provided and image recognition techniques can be used inorder to identify those objects in the images A corresponding databasecomprising such objects can, for example, comprise information of theform and shape of a femur, part of a femur, or the head of a femur.

According to an embodiment of the invention, the at least one referenceimage is an earlier acquired image of the bone.

According to an embodiment of the invention, the image is acquired afterdetection of a relevant surgery step. A relevant surgery step in thecontext of the invention can be a surgery step suitable for displacingthe bone, a part of the bone or an implant.

For instance, during final lag screw placement, displacement of thebone, a bone part or an implant can occur during the last few turns of alag screw needed to bring the screw in its final position. Typically,the force to be applied for the last few turns of a lag screw gets high,such that unwanted and undesired displacements can easily occur. Thefinal lag screw placement step can be detected by image processingmethods detecting various objects or devices and their relative positionin an image. In this way the progress and point in the workflow of asurgery can be determined, including, for instance, the step of finallag screw placement. For example, the image processor can determine whena head portion of a lag screw contacts bone or an implant on the bone byanalyzing the image and reference images. Alternatively, or in addition,the image processor can determine when a tip portion of a lag screwstarts to enter or enters into robust bone material by analyzing theimage and the reference image. For instance, robust bone material can beidentified by analyzing the texture of respective portions of the bonein the image and the reference image. If a lag screw starts to penetrateinto robust bone material, the friction between screw and bone increasesin this region and the risk of a rotation of e.g. a bone fragment inthis region rises when a force is exerted in order to turn the screw.Such situation may happen in the last turns of a lag screw, when the tipportion of the screw may enter into a robust bone material where it isfixated.

As used herein, the term “anatomical structure” refers to anything at abone and in particular to a geometrical aspect of a bone, i.e. a point,a line, an arc, a centre point, an axis, a cylinder surface, a ballsurface, or the like. For example, a geometrical aspect of a femur maybe the outer surface of the femur head, an axis defined by the neckbetween shaft and femur head, a longitudinal axis of the femur shaft, amost distal point on the bone surface, a line defined by the centrepoints of the condyles, or a line defined by the most posterior pointsat the condyles. It will be understood that the other bones provideother and/or comparable suitable geometrical aspects.

A computer program element capable of performing steps of the method maycomprise sets of instructions for identifying a current state of each ofthe detected elements. Here, “current state” means first of all aposition and orientation of the detected element. As will be describedin detail below, the position and orientation of the reference body canbe determined due to the specific distribution of fiducial markersforming the reference body. With respect to an instrument, like a gaugeor a drill or a K-wire, the position may be detected in relation to thereference body and/or to an anatomical structure. A “current state” ofan instrument may also include a deformation or bending of theinstrument. Furthermore, a “current state” may indicate the appearanceof the instrument and/or of an implant or sub-implant in the 2Dprojection image.

Based on the identified state of the detected elements, a state ofprogress of the procedure of fracture treatment may be determined. Forexample, a processing unit may execute sets of instructions so as tocompare information provided by a database with the results of thepreviously performed steps, with the database including data definingeach step out of a sequence of steps necessary to perform a fracturetreatment procedure. For example, the steps may be defined in thedatabase by the respective state of the elements which elements areinvolved in the particular step, so that information extracted from the2-D projection image can be compared with information received from thedatabase.

The step following the identified step out of the sequence of steps inthe database may be used by the processing unit to provide informationwhich step should be performed next. Such information can, for example,be shown on a monitor as information assisting in the fracturetreatment.

When comparing the database information with the identified state of thedetected elements, a processing unit, when executing sets ofinstructions of the computer program element, may determine a deviationof the current state of the element from a target state of the elementin which the element should be in the determined state of progress.Further, a degree of deviation may be determined, and the degree ofdeviation may be compared with a predetermined threshold value. In acase in which the determined deviation exceeds the threshold value, atleast a warning may be provided, but also a suggestion to undo a step.

The determined deviation may also be translated by a processing unitinto an adjustment movement of an extracorporeal handling device, i.e. ahandling device being outside of a body of a patient. For example, anadjustment movement may be a translational or a rotational movement of ahandle of an instrument in a specific direction so as to shift animplant in a longitudinal direction thereof or to tilt or rotate theimplant. The kind and degree of the adjustment movement may be opticallyor acoustically provided.

A system for displacement tracking during positioning and fixing of animplant in a bone, in particular a hip bone, is provided according to asecond aspect of the invention. The system comprises an imaging unit anda processing unit. The imaging unit is configured to provide an image ofthe bone and to provide at least one reference image. Thereby, the atleast one reference image can be an earlier acquired imaged stored in astorage unit of the imaging unit, or the reference image can be providedfrom a database of the imaging unit. The at least one reference imagecan also be provided from an external storage device and can, in thiscase, be acquired by a different imaging unit or different system at aprevious point in time. The processing unit is configured to superimposethe current state image with the at least one reference image. To thisend, the image and the at least one reference image are taken from asimilar angle range. The processing unit is further configured tovisualize the relationship between the image and the reference image inorder to allow for tracking of displacement of the bone relative to theimplant.

An imaging unit may be an imaging device of a medical device, forexample an X-ray imaging system. The imaging unit may be configured toacquire images of a surgical area. The imaging unit may comprise adisplay or a monitor for visualizing images, information, and/or text.

A processing unit may be a processing unit of a computer. The computermay be comprised in a medical device or may be connectable to a medicaldevice. The processing unit can comprise a processor, which can beconfigured to process data or to work on data delivered to theprocessor.

According to an embodiment of the invention, the image and referenceimage are intraoperative images taken at different times. Thereby, theimage and/or the reference image may be taken by the imaging unit of anabove described system.

According to a third aspect of the invention, a computer program isprovided, which, when executed by a processor, can perform themethod-steps described above.

According to a fourth aspect of the invention, a computer readablemedium is provided, on which the above described computer program isstored.

It has to be noted that embodiments are described with reference todifferent subject-matters. In particular, some embodiments are describedwith reference to method type claims whereas other embodiments aredescribed with reference to apparatus type claims. However, a personskilled in the art will gather from the above and the followingdescription that unless other notified in addition to any combination offeatures belonging to one type of subject-matter also any combinationbetween features relating to different subject-matter is considered tobe disclosed with this application.

BRIEF DESCRIPTION OF THE DRAWINGS

The aspects defined above and further aspects, features and advantagesof the present invention can also be derived from the examples of theembodiments to be described hereinafter and are explained with referenceto examples and embodiments also shown in the figures, but to which theinvention is not limited.

FIG. 1 shows a flow chart of steps of a method according to anembodiment of the invention.

FIG. 2 shows a schematic illustration of a monitor visualization of theimage and the at least one reference image according to an embodiment ofthe invention.

FIG. 3 is another schematic illustration of a monitor visualization ofthe image and the at least one reference image according to anembodiment of the invention.

FIGS. 4A and 4B show examples of a monitor visualization of an image anda reference image according to an embodiment of the invention.

FIG. 5 shows an example of an outline of a bone according to anembodiment of the invention.

FIG. 6 shows another flow chart of steps of a method according to anembodiment of the invention.

FIG. 7 shows a schematical illustration of a system according to anembodiment of the invention.

Throughout the drawings, the same reference numerals and characters,unless otherwise stated, are used to denote like features, elements,components, or portions of the illustrated embodiments. Moreover, whilethe present disclosure will now be described in detail with reference tothe figures, it is done so in connection with the illustrativeembodiments and is not limited by the particular embodiments illustratedin the figures.

DETAILED DESCRIPTION

The flow-chart in FIG. 1 illustrates method steps performed inaccordance with embodiments of the invention. It will be understood thatthe steps described may be major steps, wherein these major steps mightbe differentiated or divided into several sub-steps. Furthermore, theremight be also sub-steps between the steps. Consequently, groups of stepsmay be related to a major aspect which may also be performed separately,i.e. independent from other steps or groups of steps.

It is noted that some steps are described as being performed “ifnecessary”. This is intended to indicate that those steps may beomitted. It is in particular noted that a computer program elementaccording to an embodiment of the invention may comprise sets ofinstructions to automatically recognize if a step is necessary or not,and to automatically proceed with the next actually necessary step.

With reference to FIG. 1, the exemplary method starts with providing animage of a bone in step S1. The image may be an x-ray image.Alternatively, the image may be an ultrasound image, a magneticresonance image or any another type of image acquired in medicaldiagnostics and/or therapy. In method step S1′ a reference image of thebone is provided. The reference image may or may not be or the same typeof image as the image of the bone provided in step S1.

It is intended that the image and the reference image are taken from asimilar angle range. For instance, the deviation between the points ofview/angles under which the image and the reference image are taken canbe between 0° to 5°, but should not be significantly larger.

It should be noted that the time-wise order of provision of the imageand the reference image, that is the order of steps S1 and S1′, isarbitrary. The image can be provided first and the reference image canbe provided subsequently, or vice versa. If necessary, the image and thereference image are processed/analyzed, respectively, in subsequentoptional steps S2 and S2′. It should further be noted that the image istypically the last image taken/acquired of the site and that thereference image may be an earlier acquired image. That is, as far asacquisition of the image is concerned, the image is the last imagetaken. However, the order in which the image and the reference image areprovided within the method is arbitrary.

In steps S2 and S2′, an outline of the bone or of another object in theimage and in the reference image can be detected, respectively.Detection of an outline in an image may rely on identification ofgrayscale values of pixels, which are compared with adjacent pixelgrayscale values in the respective image so as to detect contour linesor points in the image.

In method step S3, the image and the reference image are superimposed.Superimposing the image and the reference image can, for instance bebased on a pixel-wise comparison of, for instance, the gray scale valuesof the image and the reference image. Alternatively, a comparison ofpredetermined or adjustable areas in the image and the reference imageor a comparison of clusters of pixels can be performed as a sub-step ofmethod step S3.

As a further sub-step to method step 3, optionally, the section and/orscale of the image and the reference image is adjusted. Such sub-stepcan be necessary if the detail in the reference image differs from thedetail in the image. In this case the image and/or the reference imagecan be cut to size such that the potentially “trimmed” images show thesame detail. In addition, or instead of, it might be necessary to adjusta section and/or a scale of the reference image and the image, which canalso be performed as part of a sub-step of the superimposing method stepS3.

In next method step S4 a relationship between the image of the bone andthe reference image is visualized, for instance on a computer display ora computer screen/monitor. In displaying the relationship between theimage and the reference image, differences can optionally behighlighted.

With reference to FIG. 2, a display 3 for visualizing the image of thebone as well as the reference image in order to show a relationshipbetween these images is shown. The image 1 and the reference image 2 areshown in juxtaposition, in particular next to each other. A surgeon oruser can then identify by eye differences between the image and thereference image. An illustrative example of an image and a referenceimage will be further discussed below in context of FIGS. 4A and 4B.

In FIG. 3, another example for a display 3, which can be used forvisualizing the relationship between an image and at least one referenceimage is shown. On the display 3 a combined visual representation of animage and a reference image is shown: both images comprise a femur andin the combined visual representation of the two images the image of thefemur 10 comprised in the image is shown with solid outline. The imageof the femur 20 comprised in the reference image is shown with dashedoutline. In another embodiment not shown in FIG. 3, the image of thefemur 10 comprised in the image and the image of the femur 20 comprisedon the reference image can be visualized together—that is at a time—ondisplay 3 with a specific color marking. For instance, the image of thefemur 10 can be visualized in a color differing from the color used fordisplaying the image of the femur 20. Alternatively, or in addition, adifferent shading and/or a different hatching can be used in order tovisualize the image of the femur 10 and the image of the femur 20together on display 3. The combined visual representation of femur 10and femur 20 can provide support for a surgeon or user in order todetect or track displacements of the femur. Such displacement can, forinstance, arise during final lag screw placement, when the force exertedby a surgeon in order to finally fix the screw in the bone material istypically largest during implant-fixing-workflow.

With reference to FIGS. 4A and 4B, an example of a reference image and acorresponding image are shown. FIG. 4A shows a reference image with ahip bone 4 and an implant, given by a lag screw 5. Further shown are areference body, which may be a connector 6 to a targeting instrument anda plurality of additional reference objects. The additional referenceobjects shown in FIG. 4A are markers 7, which can be of use in thedetermination of a spatial arrangement and orientation of the furtherobjects shown in the images relative to each other. To this end, theposition of objects relative to a marker or a set of markers can bedetermined and from this a spatial arrangement of the objects relativeto each other can be deduced. This can, for example, lead to thedetermination of the position of the tip of lag screw 5 within the bone4. In FIG. 4B the same surgery area is shown, containing the same objectas the reference image of FIG. 4A. However, the position of the tipportion 50 of lag screw 5 differs in the image from its correspondingposition in the reference image. The image has been acquired after a fewmore turns of the screw into the bone material have been performed. Thisrequires exertion of a force, and the visual representation of the imageand the reference image according to embodiments of the invention cansupport a surgeon to identify displacements of the implant or the bone,which occurred during operation steps performed in the time duringtaking of the reference image and the image.

In FIG. 5, an example of an outline 80 of a hip bone 8 is shown. Forinstance, by identification of grayscale values of pixels in the imageand the reference image and comparison with adjacent pixel-greyscalevalues, contour lines of a bone, e.g. a hip bone, can be detected in theimage and the reference image, respectively. From these contour lines anoutline of the bone can be derived, wherein the outline refers to thesilhouette of the bone visible before the background in the image or thereference image, respectively.

FIG. 6 illustrates steps performed in accordance with embodiments of themethod according to the invention. Similar to the first steps describedwith reference to FIG. 1, the method illustrated in FIG. 6 starts withprovision of an image in step S1 and provision of at least one referenceimage in step S1′. The order of provision of the image and the referenceimage, that is the time-wise order of steps S1 and S1′ is arbitrary. Ina next steps S1 a, the image is analyzed and primary structures depictedin the image are distinguished from non-primary structures. A primarystructure in this context is a structure of interest for the surgicalprocedure. For instance, during final lag screw placement in a hip bone,primary structures can be at least one of the hip bone, the implant andthe lag screw. A non-primary structure can be the tissue surrounding thesurgical area. It can be intended that a user or surgeon can identifywhich structures are to be treated as primary structures and whichstructures should be considered as non-primary structures. To this end,an input unit can be provided, which is configured to send or deliverthe corresponding information to a processing unit, which is configuredto perform the distinction between primary and non-primary structuresSimilar to the case of the image, the reference image can be analyzedand primary and non-primary structures can be distinguished therein inmethod step S1 a′. Steps S1 a and S1 a′ can be performed in parallel orone after the other.

In step S1 b, structures that have been identified as non-primarystructures in the image are removed from the image and only the primarystructures are kept in the image for later visual representation. Asimilar procedure is performed in method step S1 b′ for the case of thereference image. After method steps S1 b and S1 b′ have been performed,the resulting image and reference image are superimposed in step S3, anda visual representation of the image and the reference image, which isused in order to track displacements of, e.g. a bone shown in bothimages, is given in step S4.

It is understood, that the image and the reference image are taken froma similar angle range. Further, it is understood by the person skilledin the art, that the method steps explained with reference to FIG. 6 canbe combined with method steps described in connection with FIG. 1.

FIG. 7 shows an exemplary embodiment of a system 9 according to anembodiment of the invention. Substantially, necessary for performing thesteps of the method, a processing unit 100 is part of the device.

An exemplary imaging device or imaging unit 200 includes an X-raysource, and an X-ray detector 260, wherein these two units are mountedon a C-arm 220.

Furthermore, the system 9 in FIG. 7 includes an input unit 300, by meansof which for example an intended imaging direction may be manuallyentered. Further, a user can input structures, which shall be consideredas primary structures in the images. Also shown is a connection to adatabase 600, located for example in a network. The database 600 maycomprise information regarding anatomical structures, for example from3D scans of different anatomical structures, so that the imagedanatomical structure may be compared with this information so as todetermine or identify specific anatomical structures. The database mayfurther comprise information regarding a sequence of necessary and/orpossible steps of a surgical procedure. Further, the database cancomprise a storage comprising at least one or a series of earlieracquired reference images. It is noted that it is also possible toautomatically determine the progress of the surgical procedure based ondetectable aspects in an x-ray image, wherein such aspects may be ininstrument and/or implant. For instance, the distance of a tip portionof a lag screw from robust bone material or from an outline of a bonemay be automatically determined by analysing the image and the referenceimage. In such a way it can be determined or estimated how many turns ofa lag screw may still be necessary to bring, e.g. the tip portion of thescrew in contact with robust bone material. Alternatively, the tipportion may already be in contact with robust bone material and may beclose to its final position, which refers to a certain distance betweenthe tip of the screw from an outline of a bone. From a determination ofthe distances between the tip portion of the screw and e.g. an outlineof the bone the number of screw turns needed to fixate a predeterminedportion of the tip in the bone material without arriving at the outlinemay be determined.

Finally, there is an indication in FIG. 7 of an anatomical structure ofinterest 500 as well as of a reference object 64 formed by a pluralityof radiopaque spheres. Within said anatomical structure, for example abone of a patient may be located which may be subject to the describedprocedures.

While embodiments have been illustrated and described in detail in thedrawings and afore-going description, such illustrations anddescriptions are to be considered illustrative or exemplary and notrestrictive, the invention is not limited to the disclosed embodiments.

Other variations to the disclosed embodiments can be understood andeffected by those skilled in the art in practising the claimedinvention, from a study of the drawings, the disclosure and the appendedclaims. In the claims, the word “comprising” does not exclude otherelements or steps, and the indefinite article “a” or “an” does notexclude a plurality. A single processor or other unit may fulfil thefunctions of several items recited in the claims.

The mere fact that certain measures are recited and mutually differentdependent claims does not indicate that a combination of these measurescan not be used to advantage. The computer program may bestored/distributed on a suitable medium such as an optical storagemedium or a solid-state medium supplied together with or as a part ofanother hardware, but may also be distributed in other forms, such asvia the Internet or other wired or wireless telecommunication systems.Any reference signs in the claims should not be construed as limitingthe scope.

Although the invention herein has been described with reference toparticular embodiments, it is to be understood that these embodimentsare merely illustrative of the principles and applications of thepresent invention. It is therefore to be understood that numerousmodifications may be made to the illustrative embodiments and that otherarrangements may be devised without departing from the spirit and scopeof the present invention as defined by the appended claims.

1. A method for displacement tracking during positioning and fixing ofan implant in a bone, in particular in a hip bone, the methodcomprising: superimposing an image of the bone with at least onereference image, wherein the image of the bone is taken from a firstangle and the at least one reference image is taken from a second angle,wherein the difference between the first angle and the second angle isno more than 5°, and visualizing a relationship between the image of thebone and the at least one reference image for tracking a displacement ofthe bone relative to an implant.
 2. The method according to claim 1,wherein the step of visualizing comprises juxtaposing a visualrepresentation of the image of the bone with a visual representation ofthe at least one reference image.
 3. The method according to claim 1,wherein the step of visualizing comprises switching between a visualrepresentation of the image of the bone and a visual representation ofthe at least one reference image.
 4. The method according to claim 1,wherein the step of visualizing comprises combining a visualrepresentation of the image of the bone and a visual representation ofthe at least one reference image, wherein the image of the bone and theat least one reference image have different color markings.
 5. Themethod according to claim 1, wherein the image of the bone and the atleast one reference image each include at least one reference body, andwherein the step of superimposing the image of the bone and the at leastone reference image is based on the at least one reference body in theimage of the bone and in the at least one reference image.
 6. The methodaccording to claim 5, wherein the at least one reference body is one ofa bone, a bone portion, a bone shape, an implant or an additionalreference object.
 7. The method according to claim 1, further comprisinga step of: detecting an outline of the bone in the image of the bone anddetecting an outline of the bone in the at least one reference image,wherein the step of superimposing the image of the bone and the at leastone reference image is based on the detected outlines in the image ofthe bone and in the at least one reference image.
 8. The methodaccording to claim 1, wherein the step of superimposing the image of thebone and the at least one reference image is based on a pixel wisecomparison, an area wise comparison or a comparison of clusters ofpixels in the image of the bone and in the at least one reference image.9. The method according to claim 1, wherein the step of superimposingcomprises only adjusting a section and/or a scale of the at least onereference image and the image of the bone.
 10. (canceled)
 11. The methodaccording to claim 1, further comprising: distinguishing between primarystructures and non-primary structures in the image of the bone and inthe at least one reference image, and removing the non-primarystructures and depicting only the primary structures in the image of thebone and in the at least one reference image, wherein the primarystructure is at least one of a bone, a bone shape, a bone fragment, animplant, a reference body and/or an additional reference object.
 12. Themethod according to claim 1, wherein the at least one reference image isan earlier acquired image of the bone.
 13. The method according to claim1, wherein the image of the bone is acquired after a relevant surgerystep is detected, wherein a relevant surgery step is a surgery stepsuitable for displacing the bone, a part of the bone or an implant. 14.A system for displacement tracking during positioning and fixing of animplant in a bone, in particular in a hip bone, comprising: an imagingunit, and a processing unit, wherein the imaging unit is configured forproviding an image of the bone and at least one reference image, whereinthe processing unit is configured for superimposing the image of thebone with the at least one reference image, wherein the image of thebone is taken from a first angle and the at least one reference image istaken from a second angle, wherein the difference between the firstangle and the second angle is no more than 5°, and wherein theprocessing unit is further configured for visualizing a relationshipbetween the image of the bone and the at least one reference image fortracking a displacement of the bone relative to an implant.
 15. Thesystem of claim 14 wherein the image of the bone and the at least onereference image are intraoperative images taken at different times. 16.A computer program, which, when executed by a processor, performs themethods steps according to claim
 1. 17. A computer readable medium, onwhich a computer program according to claim 16 is stored.